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Preface 


This  briefing  summarizes  research  that  reviews  current  U.S.  Department  of  Defense  (DoD) 
policy  governing  the  development  of  integrated  command,  control,  communication,  and  intel¬ 
ligence  (C3I)  and  weapon  systems.  Our  focus  is  on  interoperability  and  information  assurance 
(IA)  policy  in  the  context  of  DoD  acquisition  and  requirement  policy.  This  review  was  con¬ 
ducted  to  identify  ambiguities,  conflicts,  overlaps,  and  shortfalls  in  DoD  policy  and  to  recom¬ 
mend  solutions  for  clarifying  policy  and  remedying  other  shortcomings  that  were  found. 

This  research  should  be  of  interest  to  DoD  personnel  who  are  responsible  for  formulat¬ 
ing,  reviewing,  or  implementing  DoD  policy  pertaining  to  the  development  and  upgrade  of 
interoperable  C3I  and  weapon  systems. 

This  research  was  cosponsored  by  the  Office  of  the  Under  Secretary  of  Defense  for  Acqui¬ 
sition,  Technology,  and  Logistics  (OUSD[AT&L])  and  the  Assistant  Secretary  of  the  Navy, 
Research,  Development,  and  Acquisition  (RDA)  Chief  Systems  Engineer  (CHSENG),  and 
conducted  within  the  Acquisition  and  Technology  Policy  Center  of  the  RAND  National 
Defense  Research  Institute,  a  federally  funded  research  and  development  center  sponsored  by 
the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the  Unified  Combatant  Commands,  the 
Department  of  the  Navy,  the  Marine  Corps,  the  defense  agencies,  and  the  defense  Intelligence 
Community. 

For  more  information  on  the  RAND  Acquisition  and  Technology  Policy  Center,  contact 
the  Director,  Philip  Anton.  Tie  can  be  reached  by  email  at  atpc-director@rand.org;  by  phone 
at  310-393-0411,  extension  7798;  or  by  mail  at  the  RAND  Corporation,  1776  Main  Street, 
Santa  Monica,  California  90407-2138.  More  information  about  RAND  is  available  at  http:// 
www.rand.org. 
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Introduction 


RAND 


NATIONAL  DEFENSE  RESEARCH  INSTITUTE 


IMPROVING  DOD  POLICY 
GOVERNING 
ACQUISITION  OF 
C3IAND  WEAPON  PROGRAMS 


Sponsored  by 

Assistant  Secretary  of  the  Navy  RDA  CHSENG 

OUSD(AT&L) 


This  briefing  summarizes  the  results  of  a  research  project  cosponsored  by  the  Office  of  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  (OUSD[AT&L])  and 
the  Assistant  Secretary  of  the  Navy,  Research,  Development,  and  Acquisition  (RDA)  Chief 
Systems  Engineer  (CHSENG). 

This  briefing  was  presented  to  Vitalij  Garber,  senior  technical  advisor  for  interoperability, 
OUSD(AT&L);  and  Cheryl  Walton,  director,  standards  policy  and  guidelines,  Assistant  Sec¬ 
retary  of  the  Navy,  RDA  CHSENG,  on  June  20,  2006. 

This  briefing  reflects  the  conditions  of  the  U.S.  Department  of  Defense  (DoD)  acquisi¬ 
tion  environment  at  the  time  the  analysis  was  performed  and,  unless  otherwise  noted,  does  not 
take  into  consideration  changes  that  have  occurred  since  June  2006. 
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Purpose 

•  Identify  ambiguities,  conflicts,  overlaps,  and 
shortfalls  in  current  policies  and  recommend 
solutions  in 

-  Joint  Capabilities  Integration  and  Development 
System  (JCIDS) 

-  acquisition 

-  IT  system  interoperability  and  supportability 

-  net-centric  implementation  documents  (NCIDs) 

-  information  assurance  (IA) 

•  Focus  -  interoperability  policy  statements  in  DoD 
policy 

RAND  2 

The  purpose  of  the  project  is  to  review  current  DoD  policy  governing  the  development  and 
upgrade  of  interoperable  command,  control,  communication,  and  intelligence  (C3I)  and 
weapon  systems.  Our  focus,  therefore,  is  those  elements  of  DoD  policy  that  pertain  to  the  IT 
component  of  these  programs.1  We  reviewed  this  policy  area  to  identify  ambiguities,  conflicts, 
overlaps,  and  shortfalls  in  these  policies  and  to  recommend  solutions  for  clarifying  the  ambi¬ 
guities,  mitigating  the  shortfalls,  filling  the  gaps,  and  resolving  the  conflicts  we  found  in  the 
policy  statements. 

We  examined  five  policy  areas  that  apply  to  the  interoperability  of  C3I  and  weapon  sys¬ 
tems.  We  started  by  looking  at  the  requirement-gathering  process.  The  current  process  is  gov¬ 
erned  by  requirement  policy  as  defined  in  the  Joint  Capabilities  Integration  and  Development 
System  (JCIDS)  documentation.  Next,  we  reviewed  DoD  acquisition  policy  as  determined  by 
DoD  5000-series  documentation,  as  well  as  several  recent  studies  of  the  acquisition  system 
that  identify  where  current  policy  may  be  deficient  and  can  be  improved.  Following  acquisi¬ 
tion,  we  turned  our  focus  to  examining  interoperability  and  supportability  policy  that  has  been 
developed  and  recently  revised  specifically  for  IT  systems.  Next,  we  examined  a  new  policy 
area  designed  to  enhance  and  ensure  the  effective  integration  of  global  information  grid  (GIG) 
component  programs  so  they  perform  effectively  as  an  end-to-end  (E2E)  system  of  systems. 
Policies  in  this  new  area  are  in  the  net-centric  implementation  documents  (NCIDs).  Finally, 
we  examine  information  assurance  (IA)  policy. 
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Specifically,  we  focused  on  weapon  programs  with  high  IT  content. 
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Background 

•  What  is  interoperability? 

•  Several  interoperability  definitions  exist 

•  DoD  uses  architectural  approach  to  defining  and  achieving 
interoperability 

-  Current  bottom-up  process  inhibits  efficient  development 
of  a  single,  integrated  architecture  that  summarizes  DoD 
interoperability  requirements 

•  Program  manager  task  has  become  more  challenging 
because  of  the  growing  complexity  of  interoperability 
guidance 


RAND  3 

Interoperability  in  the  broadest  sense  is  a  measure  of  the  degree  to  which  various  organizations 
or  individuals  can  operate  together  effectively.  The  U.S.  Joint  Chiefs  of  Staff  (JCS)  define 
interoperability  as  the  ability  of  systems,  units,  or  forces  to  provide  services  to  and  accept  ser¬ 
vices  from  other  systems,  units,  or  forces  and  to  use  the  services  so  exchanged  to  enable  them 
to  operate  effectively  together  (JCS,  1994  [1999]). 

Interoperability  can  be  achieved  by  standardization,  integration,  and  cooperation  in  the 
development,  configuration,  training,  and  use  of  systems  to  support  operations  and  to  com¬ 
mand  and  control  joint  and  coalition  forces. 

An  essential  element  of  interoperability  is  system  interoperability,  which  is  the  ability  of 
two  or  more  systems  or  components  to  exchange  information  and  to  use  the  information  that 
has  been  exchanged  (IEEE  Computer  Society,  1990). 

Joint  and  coalition  interoperability  needs  for  specific  systems  and  military  units  are  con¬ 
text  dependent.  To  capture  the  operational,  system,  and  technology  contexts  for  new  system 
development,  DoD  has  adopted  an  architectural  approach.  An  architectural  approach  is  also 
dictated  by  U.S.  law  (based  on  the  former  Clinger-Cohen  Act)  (Public  Law  104-106).  The 
architecture  discloses  operational,  system,  and  technical  information  and  dependencies  needed 
to  demonstrate  interoperability.  Consequently,  current  DoD  policy  states  that  program  man¬ 
agers  (PMs)  need  specific  architecture  products  to  analyze  the  interoperability  and  supportabil- 
ity  of  warfighting  capability.  Approval  of  the  analysis  is  used  in  program  milestone  review  deci¬ 
sion.  Thus,  interoperability  requirements  of  a  new  system  in  DoD  can  be  described  through 
the  use  of  operational,  system,  and  technical  architectural  views  or  products.  The  PM’s  task, 
however,  has  recently  become  more  complex  and  challenging  because  of  additional  amplifying 
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interoperability  guidance  that  DoD  has  established  and  because  of  the  growing  complexity  of 
this  guidance. 

The  current  DoD  architectural  approach  to  interoperability  has  shortcomings  for  the  PM 
and  the  department  as  a  whole.  PMs  frequently  have  to  develop  their  own  architecture  prod¬ 
ucts  for  individual  systems  because  the  current  DoD  architecture  process  has  not  produced 
the  context  architecture  products  they  need  to  conduct  the  analysis.  Additionally,  DoD  uses 
a  bottom-up  architectural  development  process.  This  bottom-up  approach  can  result  in  much 
duplication  of  effort  across  the  entire  acquisition  system  and  thus  make  the  development  of  a 
single,  integrated  architecture  that  summarizes  DoD  interoperability  requirements  difficult  to 
achieve.  After  completion  of  this  research,  DoD  started  development  of  a  new  architectural 
approach  for  GIG  that  includes  top-down  guidance  and  architecture  product  data  standards, 
which  may  reduce  the  burden  on  PMs.  (See  Appendix  A  for  further  discussion  of  integrated 
architectures). 
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Approach 

•  Examine  applicable  DoD  policies  and  decompose  them  into 
key  elements 

•  Summarize  and  diagram  policies,  identifying  major  provisions 
and  interrelationships 

-  Management  structures 

-  Specified  documentation  program  managers,  others 
called  upon  to  deliver 

-  Subject-matter  topics 

-  Process  specifications 

•  Identify  ambiguities,  conflicts,  and  shortfalls  in  the  policies 

-  Missing  elements  crucial  to  policy  implementation 


RAND  4 

In  this  analysis  of  DoD  policy,  we  decomposed  policies  into  their  key  moving  parts.  Appendix 
B  lists  the  policies  we  examined.  We  summarized  these  policies  and,  in  particular,  identified 
their  major  provisions,  management  structures,  who  is  responsible  for  what  aspects  of  policy 
implementation,  the  subject  matter  covered,  and  the  processes  that  are  specified  to  accomplish 
the  policy  goals.  As  a  part  of  this  analysis  and  decomposition,  we  also  performed  a  cross¬ 
policy  analysis  to  identify  ambiguities,  conflicts,  shortfalls,  and  missing  elements  that  might 
be  crucial  to  policy  implementation.  As  the  following  slides  show,  we  explored  selected  areas  in 
greater  detail  to  illuminate  key  issues. 


CHAPTER  TWO 


Requirement  Policy 


Outline 


•  JCIDS 

•  Acquisition 

•  Interoperability 

•  NCIDs 

•  IA 

•  Summary 
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The  JCIDS  policy  statements  are  contained  in  the  DoD  3170  series  of  documents.  We  sum¬ 
marize  our  findings  for  JCIDS  policy  documentation  in  the  next  two  slides. 
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JCIDS  -  3170  Series 

•  Description  of  JCIDS  processes  and  products  can  be  ambiguous 

-  CBA  and  acquisition-related  processes  never  discussed  or 
diagrammed  jointly 

-  Not  clear  what  products  are  expected  at  each  stage  of  CBA  (FAA, 
FNA,  FSA),  much  less  what  those  products  should  look  like 

-  Unclear  how  CBAs  relate  to  programs  already  in  progress  or 
deployed  at  time  JCIDS  was  created 

-  Recommendation :  Address  above  in  next  policy  revision,  using 
“best”  CBA  and  systems  documentation  submitted  to  date  as 
examples 

•  Example  end-to-end  process  diagram  on  next  slide 

•  Traceability  of  capability  needs  across  documents  unclear 

-  Indirect  linkage  between  needs  coming  from  COCOMs  (IPLs, 
lessons  learned,  etc.),  JCDs,  and  other  documents 

-  Indirect  linkages  between  requirements  in  operational  documents 
(JCD,  ICD)  and  program  development  documents  (CDD,  CPD) 

-  Recommendation :  Directly  require  linkages  from  COCOM  needs  to 
JCDs  and  other  documents  and  audit  whether  COCOM  needs  are 
being  addressed  on  a  regular  schedule 

RAND 


We  found  two  major  issues  with  the  JCIDS  process  with  respect  to  interoperability  policy.  The 
first  is  that  JCIDS  processes  and  products  are  described  ambiguously.  The  3170  series  describes 
two  separate  processes — one  for  performing  capability-based  assessments  (CBAs)  and  one  for 
system  acquisition  decisions.  Though  the  3170  series  clearly  states  that  the  acquisition  process 
is  described  in  the  5000  series,  it  also  states  that  there  is  a  relationship  between  the  two.  The 
two  are  never  discussed  or  diagrammed  jointly,  making  it  difficult  to  determine  exactly  how 
CBA  products  relate  to  acquisition-related  products.  Within  the  CBA  process,  it  is  not  clear 
what  products  are  expected  as  deliverables  from  each  stage  of  the  analysis  (functional  area  anal¬ 
ysis  [FAA],  functional  needs  analysis  [FNA],  and  functional  solution  analysis  [FSA]),  much 
less  what  the  content  of  those  products  should  be.  In  response,  we  recommend  clarifying  how 
CBA  and  acquisition  processes  relate  to  each  other  and  describing  the  products  expected  at 
each  stage  of  the  CBA  process.  We  also  recommend  that  the  JCS  provide  a  set  of  examples  for 
the  best  CBA  and  system  documentation  submitted  to  date.  The  examples  can  serve  to  inform 
PMs  on  acceptable  means  of  satisfying  the  JCIDS  policy  guidance. 

The  second  major  issue  is  a  lack  of  clear  traceability  across  JCIDS  and  other  warfighting 
needs  documents.  There  is  only  an  indirect  link  between  the  needs  coming  from  command¬ 
ers  (integrated  priority  lists  [IPLs],  lessons-learned  documents)  and  JCIDS  documents  such  as 
joint  capability  documents  (JCDs).  The  former  are  never  mentioned  as  formal  inputs  to  the 
JCIDS  process,  and  there  are  no  provisions  to  trace  JCIDS  requirements  to  warfighting  needs 
documents.  Within  the  JCIDS  acquisition  process,  there  are  only  indirect  links  between  the 
requirements  in  the  early  operational  documents  (JCDs,  initial  capability  documents  [ICDs]), 
and  the  requirements  in  the  program  development  documents  (capability  development  docu- 
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ments  [CDDs],  capability  production  documents  [CPDs]).  Again,  there  is  no  provision  to 
trace  requirements  and,  in  particular,  interoperability  requirements  between  these  documents 
and  identify  which  requirements  change  as  successive  JCIDS  documents  are  developed.  In 
response,  we  recommend  requiring  linkages  from  combatant  command  (COCOM)  needs 
documents  to  JCDs  and  other  JCIDS  documents  that  specifically  pertain  to  interoperability. 
We  also  recommend  having  the  JCS  perform  audits  on  a  regular  basis  to  determine  whether 
priority  COCOM  needs  and  interoperability  shortfalls  are  being  addressed,  whether  through 
nonmateriel  or  materiel  means. 
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Conceptual  JCIDS  Relationship  to  the  Acquisition  Process 


Aik  family  of  Joint  Future  Concepts,  CONOPS,  joint  tasks,  integrated  architectures,  capability  road  maps 


Acquisition  steps 


SOURCES:  JCS  (2005a,  figures  A-1  and  A-2;  2005b,  Figure  A-1  and  tables  D-1,  E-1,  F-1,  and  G-1) 
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This  slide  is  a  conceptual  depiction  of  what  the  relationships  between  the  JCIDS  CBA  and 
acquisition  processes  might  look  like.  As  shown,  the  main  link  between  the  CBA  process 
and  the  acquisition-related  process  appears  to  be  that  ICDs  should  reflect  the  solutions  identi¬ 
fied  through  the  FSA  and  verified  through  the  program  initiation  agreement  (PIA). 

The  slide  also  shows  the  documents  that  the  3170  series  lists  as  being  inputs  for  each 
JCIDS  document,  as  well  as  output  documents  (i.e.,  documents  that  will  use  the  JCIDS  docu¬ 
ment  as  an  input).  Some  documents  are  considered  to  be  overarching  (DoD  Strategic  Guid¬ 
ance,  Joint  Future  Concepts  and  concept  of  operations  [CONOPS],  joint  tasks,  integrated 
architectures,  capability  road  maps)  (DoD,  2004a),  and  thus  influence  the  preparation  of  all 
JCIDS  documents.  As  noted  earlier,  there  are  no  requirements  to  consider  warfighting  needs 
documents  (lessons  learned,  IPLs)  in  creating  JCIDS  documents. 

We  will  see  in  the  next  chapter  that  the  acquisition  policy  documents  offer  no  further 
guidance  on  how  the  JCIDS  process  relates  to  the  acquisition  process. 


CHAPTER  THREE 


Acquisition  Policy 


Outline 


•  JCIDS 

•  Acquisition 

•  Interoperability 

•  NCIDs 

•  IA 

•  Summary 
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We  now  look  at  acquisition  policy  contained  in  the  5000-series  documents. 
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Acquisition  (5000  Series) 

•  DoDI  5000  has  not  been  updated  in  some  time,  while  specific 
IT  system  acquisition  guidance  has  evolved  more  rapidly 

-  has  a  single  system  focus 

-  discusses  interoperability  in  very  general  terms 

-  does  not  describe  Defense  Acquisition  Board  Capability  Area 
Review  (DAB  CARs) 

-  does  not  align  acquisition  with  QDR  portfolio  management 
system 

•  DoD  Acquisition  Guidebook  (DoD,  2004a)  contains  more 
current  guidance  but  is  not  directive 

-  contains  a  global  map  of  related  policies  and  guidance, 
but  is  not  always  clear  about  which  policies  are  directive 

•  DoD  5000-series  guidance  is  being  updated  and  due  to  be 
released  later  in  2007 
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The  5000  series  of  documentation  governs  acquisition  policy  at  the  top  level.  DoD  Instruction 
5000.2  (OUSD[AT&L],  2003),  has  not  been  updated  since  May  12,  2003,  while  more  specific 
IT  system  acquisition  guidance  and  interoperability  policy  has  evolved  much  more  rapidly  and 
carries  more  current  dates.  A  few  important  points  to  make  about  the  5000  series  are  that  this 
series  still  has  a  single-system  focus  and  concentrates  on  the  acquisition  system  for  single  pro¬ 
grams.  It  discusses  interoperability  but  only  in  general  terms.  This  more  generic  approach  to 
interoperability  is  not  necessarily  a  drawback  of  the  policy,  given  the  more  specific  policies  that 
do  address  interoperability.  However,  the  5000  series  does  not  address  the  Defense  Acquisition 
Board  (DAB)  capability  area  review  (CAR)  process  that  was  instituted  in  recent  years  to  look 
at  the  acquisition  of  systems  within  a  system-of-system  context.  These  new  acquisition  man¬ 
agement  elements  have  been  devised  in  an  effort  to  improve  the  synchronization  of  programs 
that  need  to  be  aligned  to  provide  system-of-system  capabilities.  Therefore,  it  is  important  to 
understand  how  they  fit  into  the  acquisition  guidance  stated  in  the  DoD  5000  series. 

The  5000  series  also  does  not  address  the  recent  Quadrennial  Defense  Review  (QDR)— 
mandated  capability  portfolio  management  (CPM)  process  and  the  four  experiments  that 
are  now  being  undertaken  in  the  department  to  do  capability  management  using  a  portfolio 
approach  across  a  set  of  programs.  The  four  CPM  experiments  are  in  the  areas  of  joint  com¬ 
mand  and  control;  joint  network  operations;  joint  logistics;  and  battlespace  awareness  or  intel¬ 
ligence,  surveillance,  and  reconnaissance. 

It  should  be  noted  that  the  DoD  acquisition  guidebook  is  available  online  (DoD,  2004a) 
and  provides  links  to  a  much  broader  set  of  policies.  However,  the  guidebook  is  not  directive. 
It  is  discretionary  and  not  mandatory  guidance,  and  it  contains  links  to  and  copies  of  many 
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policies,  some  of  which  are  not  directive  or  are  in  draft  form,  so  it  can  be  confusing  for  PMs 
to  use. 

DoD  5000-series  guidance  is  being  updated.  The  revision  of  DoDI  5000.2  is  under  way 
in  the  second  quarter  of  FY  2007.  The  update  of  DoDD  5000.1  (see  DoD,  2006b)  is  expected 
to  follow  the  revision  of  DoDI  5000.2. 
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Program  Acquisition  and  Management  Chalienges 

•  DoD  programs  have  increasing  IT  content  -  computer 
hardware  and  software 

•  Key  DoD  programs  with  high  IT  content  have  encountered 
significant  cost,  schedule,  and  performance  problems 

-  higher  technology  risks  than  originally  estimated  for  key 
components 

-  Joint  Tactical  Radio  System  (JTRS) 

-  satellite  programs 

•  SBIRs 

•  FIA 

•  AEHF 

•  TSAT 

-  Army  Future  Combat  System  program 

•  Joint  interoperability  and  IA  requirements  have,  in  many 
cases,  been  key  design  drivers 

-  e.g.,  JTRS  waveforms,  DITSCAP  certification,  software 
assurance,  etc. 
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Some  DoD  programs  have  suffered  acquisition  setbacks  and  acquisition  management  chal¬ 
lenges  within  the  past  five  or  six  years.  These  programs,  which  have  suffered  the  most  serious 
cost  growth  and  schedule  slips,  appear  to  have  increasing  IT  content — that  is,  in  terms  of  total 
acquisition  cost,  an  increasing  percentage  of  the  program  deliverable  is  computer  hardware  and 
software. 

Key  DoD  programs  with  high  IT  content  that  have  encountered  significant  problems  are 
some  of  the  core  programs  for  the  next  generation  of  C3I  systems.  These  problematic  programs 
include  the  Joint  Tactical  Radio  System  (JTRS)  (Feickert,  2005)  and  a  whole  assortment  of 
satellite  programs,  including  Space-Based  Infrared  System  (SBIRS),  the  Future  Imagery  Archi¬ 
tecture  (FIA)  system,  the  Advanced  Extremely  High  Frequency  (AEHF)  program,  and  the 
Transformational  Satellite  (TSAT)  system  (Hura  et  ah,  forthcoming).  Another  program  with 
high  IT  content  that  has  suffered  some  significant  cost,  schedule,  and  performance  issues  is  the 
U.S.  Army’s  Future  Combat  System  (FCS)  (Bowman,  2005).  These  programs  are  just  a  few 
examples  of  programs  with  high  IT  content  that  have  encountered  acquisition  issues.  Many 
other  high— IT  content  programs  exist. 

The  common  theme  among  all  of  these  programs,  and  perhaps  a  source  of  their  prob¬ 
lems,  is  that  technology  risk  appears  to  have  been  higher  than  originally  estimated  for  many 
key  components.  Some  of  the  most  recent  examples  of  this  are  the  laser  communication  com¬ 
ponents  associated  with  TSAT  and  the  technology  challenges  associated  with  producing  and 
maturing  those  technologies. 

Another  important  source  of  technology  risk  and  program  performance  risk  is  joint 
interoperability  and  IA  requirements.  In  many  cases,  joint  interoperability  and  IA  require- 
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ments  have  been  key  design  drivers  that  have  resulted  in  more  requirements  and  more  complex 
requirements  being  levied  on  these  programs.  Examples  of  such  cases  include  the  original  32 
software- designed  waveforms  that  the  JTRS  program  was  supposed  to  implement.  A  second 
example  is  the  complexity  of  the  DoD  Information  Technology  Security  Certification  and 
Accreditation  Process  (DITSCAP)  for  IA.  A  third  and  recent  example  is  software  assurance 
issues  associated  with  many  programs  in  which  higher-cost,  domestic  software  development 
teams  have  to  be  used  because  less  costly  overseas  software  development  poses  serious  software- 
assurance  issues. 
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Findings  of  Recent  Reviews  of  the 
DoD  Acquisition  System 

•  DSB  summer  study  on  defense  transformation  -  the 
acquisition  system 

-  is  inflexible  and  risk  averse 

-  frequently  does  not  deliver  systems  at  cost  and  on  time 

-  does  not  allow  for  rapid  technology  insertion 

-  does  not  adequately  address  technology  risk  factors 

•  Defense  Acquisition  Performance  Assessment 
(DAPA) 

-  System  is  slow  and  overly  complex 

-  DoD  5000  sets  Milestone  B  too  early 

•  Before  system  design  and  technology  are  sufficiently  mature  to 
establish  high  confidence  cost,  schedule,  and  performance  thresholds 

-  DoD  lacks  clearly  definable  measures  of  technology 
readiness 

RAND 

Two  recent  studies  have  reviewed  the  DoD  acquisition  system.  The  first  was  the  Defense  Sci¬ 
ence  Board  (DSB)  summer  study  on  defense  transformation  (DSB  and  OUSD[AT&L],  2006). 
The  DSB  study  concluded  that  the  current  acquisition  system  is  inflexible  and  risk  averse. 
These  characteristics  result,  too  frequently,  in  systems  that  breach  cost  and  schedule  goals.  In 
addition,  the  current  acquisition  system  does  not  allow  for  rapid  technology  insertion  and  does 
not  adequately  address  technology  risk. 

The  second  major  study  was  the  Defense  Acquisition  Performance  Assessment  (DAPA) 
(Assessment  Panel  of  the  Defense  Acquisition  Performance  Assessment  Project  for  the  Deputy 
Secretary  of  Defense,  2006).  The  DAPA  conclusions  were  similar  to  the  DSB  acquisition  study 
findings,  but  the  DAPA  study  identified  specific  sources  for  defense  acquisition  shortcom¬ 
ings.  For  example,  the  DAPA  study  found  that  DoD  5000  policy  sets  MS  B  too  early  in  the 
acquisition  process.  DAPA  asserts  that  MS  B  is  scheduled  before  a  system  is  mature  enough  in 
its  design  and  technology  choices  to  enable  high-confidence  cost,  schedule,  and  performance 
thresholds  to  be  set.  In  addition,  the  DAPA  study  noted  that  DoD  lacked  clearly  definable 
technology  readiness  metrics  or  technology  maturity  metrics,  thus  further  hampering  its  abil¬ 
ity  to  specify  a  more  appropriate  placement  for  MS  B  in  the  acquisition  cycle. 

These  findings  point  toward  a  need  to  define  technology  risk  to  incorporate  the  multiple 
sources  from  which  such  risk  can  stem.  Our  examples  on  the  previous  slide  show  that  tech¬ 
nology  risk  can  have  roots  in  technology  maturation,  as  well  as  other  sources  such  as  security 
considerations.  Once  an  appropriate  definition  of  technology  risk  has  been  established  and 
the  metrics  to  measure  it  have  been  reviewed  and  agreed  to,  acquisition  policy  can  include 
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these  establishment  technology  readiness  levels  along  with  associated  time-phased  goals  to  help 
address  technology  risk. 
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Recommended  Changes  to  DoD  5000  Series 
Guidance  Specific  to  High-iT  Content  Programs* 

•  Establish  TRLs  for  key  GIG  functional  areas  that  affect 
interoperability 

-  Waveform,  communications  protocol,  and  IA  functions 

-  Quality  of  Service  signaling  and  protocols,  etc. 

•  Use  GIG  TRLs  in  reviews  of  GIG  and  GIG-related  programs 

-  Update  GIG  TRLs  based  on  learning  in  program  reviews 

•  Ensure  that  system  designs  are  reviewed  by  independent 
technical  experts  at  the  appropriate  system  commands 
(SYSCOMs) 

•  Delay  Milestone  B  to  program  preliminary  design  review 

•  As  an  adjunct  to  5000,  develop  a  Web-based  global  map  of 
acquisition  guidance 

-  that  is  updated  regularly 

-  that  only  points  to  current  authoritative  directive  policy 

RAN  D  ‘Recommendations  consistent  with  DAPA  and  2005  DSB  summer  study  12 

By  synthesizing  the  findings  from  the  DSB  and  DAPA  studies,  we  can  glean  a  number  of  rec¬ 
ommendations  for  improving  DoD  5000-series  guidance  and,  specifically,  how  to  improve 
the  way  in  which  that  guidance  relates  to  high-IT  content  programs  and  how  this  guidance 
addresses  interoperability.  While  neither  the  DSB  nor  DAPA  study  contains  these  specific  rec¬ 
ommendations,  these  suggested  improvements  are  consistent  with  the  themes  established  in 
both  studies.  The  recommendations  we  outline  below  pertain  to  high-IT  content  programs. 

Our  first  recommendation  is  that  DoD  should  establish  technology  readiness  levels 
(TRLs)  for  key  GIG  functional  areas  that  are  important  for  interoperability,  such  as  waveform 
functions,  communication  protocols,  IA  functions,  quality-of-service  (QoS)  signaling  and  pro¬ 
tocols,  and  other  areas.  New  guidance  should  specify  that  the  GIG  TRLs  be  used  in  reviews  of 
GIG  and  GIG-related  programs.  In  addition,  new  guidance  should  specify  that  the  TRLs  be 
updated  based  on  the  learning  acquired  in  program  reviews.  Technology  changes  and  matures, 
so  these  TRLs  should  be  changed  and  updated  as  time  goes  on;  specifying  an  ongoing  updat¬ 
ing  procedure  will  help  ensure  that  the  TRLs  remain  relevant  and  current. 

Another  important  recommendation  is  to  include  guidance  in  the  DoD  5000  series  that 
ensures  that  independent  technical  experts  review  system  designs,  especially  those  aspects  of 
the  design  that  relate  to  system  interoperability  or  net-centricity.  In  some  cases  in  which  very 
specialized  technologies  are  being  developed  or  employed,  finding  experts  may  be  difficult,  so 
the  DoD  5000-series  guidance  should  incorporate  a  degree  of  flexibility  to  accommodate  such 
circumstances,  but,  at  the  same  time,  the  guidance  should  encourage  all  reasonable  efforts  to 
conduct  reviews  of  programs  by  independent  experts.  The  value  of  such  independent  reviews 
is  exemplified  by  the  fact  that  one  major  command  in  the  Navy  has  already  adopted  an  inde- 
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pendent  review  policy,  though  the  DoD  5000  series  does  not  yet  include  such  a  requirement 
or  recommendation. 

Our  third  recommendation  is  that  the  DoD  delay  MS  B  to  the  program’s  preliminary 
design  review  (PDR).  At  PDR,  the  program’s  system  design  is  generally  stable  and  technologi¬ 
cal  maturity  has  progressed  to  enable  the  specification  of  attainable  cost,  schedule,  and  perfor¬ 
mance  thresholds.  This  recommendation  is  consistent  with  the  DAPA  findings. 

Finally,  we  recommend  that  DoD  develop  a  Web-based,  global  map  of  acquisition  guid¬ 
ance  as  a  supplement  to  the  hard-copy  5000-series  documentation.  The  global  map  should  link 
the  5000-series  guidance  with  all  of  the  key  underlying  policy  guidance.  The  material  on  the 
Web  site  should  be  current  and  include  only  authoritative  directive  policy.  Maintaining  cur¬ 
rency  will  require  that  the  material  on  the  Web  site  be  updated  when  any  underlying  policy  is 
updated.  The  site  should  include  an  interactive  capability  so  that  a  PM  can  very  quickly  search 
for  key  policy  areas  such  as  interoperability  and  find  all  of  the  current  policies  that  must  be 
satisfied  and  implemented  for  that  specific  policy  area. 
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In  this  chapter,  we  turn  our  attention  to  interoperability  policy.  In  the  following  slides,  we 
present  a  variety  of  views  of  global  maps  of  DoD  interoperability-related  policy  documents. 
Our  intent  is  to  illustrate  the  relationships  of  the  policy  documents  by  showing  the  organiza¬ 
tions  responsible  and  the  areas  that  the  policies  cover. 
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I  I  Other  DoD 

We  begin  with  a  slide  that  shows  the  multiplicity  of  DoD  policy  documents  that  address 
interoperability. 

A  timeline  by  quarter,  starting  in  1997  and  ending  in  2006,  is  shown  at  the  bottom  of 
the  slide.  Policy  documents  related  to  interoperability  are  shown  in  labeled  rectangles  above  the 
timeline,  according  the  release  date  of  the  document.  Per  the  legend  on  the  slide,  the  colors  of 
the  rectangles  show  the  type  of  document. 

This  slide  shows  that  there  is  a  dramatic  increase  in  the  number  of  policy  documents 
related  to  interoperability  that  have  been  issued  since  the  first  quarter  of  2002,  with  11  DoD 
directives  and  instructions  alone  in  2004.  Clearly,  carrying  out  interoperability  policy  has 
required  PMs  to  be  aware  of  many  more  policy  documents  in  recent  years.  Furthermore,  even 
if  all  of  the  policy  documents  were  consistent  and  redundant  in  defining  interoperability  policy 
or  were,  in  fact,  helpful  in  providing  more  detailed  guidance  on  existing  policies,  the  PMs  serv¬ 
ing  since  2002  still  have  had  to  comb  through  a  greater  volume  of  literature  to  ascertain  their 
responsibilities  regarding  interoperability  policy.  We  will  show  later  that  the  PM’s  task  is  actu¬ 
ally  more  complex  because  the  interoperability  policies  are  neither  redundant  nor  consistent. 
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Constructing  a  Global  Map  of  DoD  Responsibilities 


RAND  I  I  JCS  documents  Non-DoD  or  government 


|j; - I  Other  DoD 

We  reviewed  the  list  of  interoperability-related  policy  documents  shown  in  the  previous  slide 
to  determine  which  agencies  had  responsibility  for  which  aspects  of  interoperability  policy.  The 
simplified  example  in  this  slide  illustrates  our  analysis  methodology. 

The  row  of  circles  at  the  bottom  shows  interoperability-related  topic  areas  that  are  dis¬ 
cussed  in  one  or  more  of  the  five  policy  documents.  This  list  was  generated  by  first  reviewing 
the  documents  and  then  synthesizing  the  collection  of  areas  addressed  by  the  documents. 
These  topic  areas  are  spectrum,  training,  certification,  testing,  IA/critical  infrastructure  pro¬ 
tection  (CIP),  standards,  acquisition,  GIG,  architectures,  net-ready  key  performance  parame¬ 
ters  (NR-KPPs),  and  information  support  plans  (ISPs).  While  the  synthesized  list  does  address 
different  aspects  of  interoperability,  the  different  areas  in  the  list  are  not  necessarily  mutually 
exclusive  and  were  not  designed  to  be.  Rather,  our  intent  is  to  convey  the  breadth  and  depth  of 
discussions  related  to  interoperability  in  the  policy  documents.  Each  circle  is  a  different  color, 
and  the  lines  emanating  from  a  particular  circle  are  color-coordinated  with  the  circle  so  the 
reader  can  easily  ascertain  which  topic  area  is  mentioned  in  the  policy  documents.  A  solid  line 
linking  a  particular  topic  area  to  a  particular  document  indicates  that  that  policy  document 
addresses  that  topic  area. 

This  slide  shows  a  view  of  five  specific  documents  that  either  state  policies  or  implement 
policies:  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  (CJCSI)  3170. 01E  (JCS,  2005a), 
CJCSI  6212.01D  (JCS,  2006  [2007]),  DoDI  5000.2  (OUSD[AT&L],  2003),  DoDI  4630.8 
(DoD,  2004g),  and  DoDD  8100.1  (DoD,  2002c  [2003]).  These  five  documents  are  policy 
statements  or  instructions  that  implement  key  policies  involving  interoperability  and  the  GIG. 
The  documents  are  shown  in  the  five  labeled  rectangles  in  the  middle  of  the  slide.  The  colors 
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of  the  rectangles  indicate  the  type  of  document  and  follow  the  same  color-coded  legend  as  in 
the  previous  slide. 

Organizations  that  the  five  policy  documents  mention  by  name  are  shown  in  the  row 
of  circles  above  the  rectangles.  Reading  from  left  to  right  along  this  second  row  of  labeled 
circles,  the  organizations  include  the  military  services  and  various  defense  agencies  and 
organizations — including  the  combatant  commanders,  joint  staff  (JS),  Defense  Information 
Systems  Agency  (DISA),  National  Security  Agency  (NSA),  Defense  Intelligence  Agency  (DIA), 
Assistant  Secretary  of  Defense  for  Networks  and  Information  Integration  (ASD[NIIj),  DoD 
chief  information  officer  (CIO),  USD(AT&L),  USD(C),  Defense  Office  of  Testing  and  Eval¬ 
uation  (DOT&E),  USD(Policy),  USD(I),  and  the  National  Geospatial-Intelligence  Agency 
(NGA).  A  solid  black  line  linking  a  document  to  a  specific  agency  indicates  that  the  policy 
document  assigns,  by  name,  that  specific  agency  to  at  least  one  policy  responsibility. 

The  ovals  along  the  top  of  the  diagram  show  groups  of  organizations  that  are  tasked  with 
a  specific  responsibility  in  at  least  one  of  the  documents.  Therefore,  these  groups  are  not  the 
authors  of  these  documents,  but  they  are  the  entities  that,  as  a  result  of  one  of  these  policy 
documents,  were  tasked  to  execute  some  action  or  assume  a  responsibility.  A  solid  black  line 
linking  a  policy  document  to  a  group  of  organizations  means  that  the  document  mentions  the 
group  as  responsible  for  a  specific  policy.  Dashed  black  lines  from  an  oval  to  circles  in  the  row 
below  the  ovals  indicate  that  the  specific  organization  represented  by  the  circle  is  a  member  of 
the  group  represented  by  the  oval.  Therefore,  the  solid  lines  connecting  the  circles,  rectangles, 
and  ovals  show  direct  relationships  contained  in  the  policy  documents.  The  dashed  lines  indi¬ 
cate  implied  relationships.  We  illustrate  these  two  types  of  relationships  here. 

Some  of  the  policy  documents  do  not  mention  specific  organizations  but  do  refer  to  them 
as  a  group.  For  example,  some  documents  include  a  global  reference  such  as  “Department  of 
Defense  departments”  or  “all  defense  agencies.”  The  diagram  shows  these  global  references 
with  a  solid  line  from  the  document  to  the  global  reference  oval,  followed  by  dashed  lines  from 
the  global  reference  to  the  specific  entities  that  are  members  of  that  particular  global  group.  It 
is  important  to  note  that  this  view  of  DoD  responsibilities  does  not  necessarily  show  all  mem¬ 
bers  of  each  global  group.  It  shows  only  organizations  that  are  specifically  named  at  some  point 
in  one  or  more  of  the  five  documents.  There  may  be  many  more  specific  entities  that  are  mem¬ 
bers  of  a  global  group,  and  all  members  of  each  global  group,  whether  or  not  the  member  has 
been  mentioned  by  name  in  any  policy  document,  do  have  the  responsibility  given  to  a  global 
group  by  a  policy  document. 

DoDI  5000.2  (OUSD[AT&L],  2003)  is  an  example  that  illustrates  this  type  of  implied 
responsibility.  DoDI  5000.2  does  not  name  NSA,  DIA,  or  the  Defense  Advanced  Research 
Projects  Agency  (DARPA)  as  the  specific  entities  that  hold  particular  responsibilities,  but  the 
document  does  include  statements  that  hold  all  of  the  defense  agencies  accountable  for  par¬ 
ticular  responsibilities.  To  illustrate  this  global,  implied  assignment  of  responsibility,  the  slide 
shows  a  solid  line  from  DoDI  5000.2  to  the  “Defense  agencies”  oval  on  the  top  row.  Dashed 
lines  are  shown  connecting  the  “Defense  agencies”  oval  to  the  specific  organizations  in  the 
circles  on  the  second  row  that  are  defense  agencies,  including  NSA  and  DIA.  The  diagram  does 
not  show  DARPA  as  a  defense  agency  because  none  of  the  five  policy  documents  specifically 
mentions  DARPA.  There  is,  of  course,  then,  no  dashed  line  that  connects  the  “Defense  agen¬ 
cies”  oval  with  a  “DARPA”  circle.  DARPA,  however,  will  still  be  accountable  for  the  responsi¬ 
bilities  that  DoDI  5000.2  assigns  to  all  defense  agencies. 
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All  of  the  documents  contain  some  general  discussion  of  responsibilities  that  apply  to  the 
heads  of  DoD  components  or  to  other  DoD  components.  DoD  components  refers  to  all  DoD 
agencies  and  the  military  services.  Therefore,  in  this  way,  the  five  documents  have  some  refer¬ 
ence  of  generic  responsibilities  for  heads  of  DoD  components  and  other  DoD  components.  The 
slide  shows  the  presence  of  these  general  discussions  with  the  notation  adjacent  to  the  “Heads 
or  other  DoD  components”  oval. 


26  Navy/OSD  Collaborative  Review  of  Acquisition  Policy  for  DoD  C3I  and  Weapon  Programs 


Global  Map  of  DoD  Interoperability-Related  Policy 

Documents 


RAND 


DoD  directives  &  instructions  [ 
JCS  documents  l 


Other  DoD 


]  DoD  CIO  memorandum 
]  Non-DoD  or  government 


16 


This  slide  shows  a  more  complete  picture  of  DoD  interoperability-related  policy  documents.  In 
this  view,  the  circles  along  the  bottom  show  the  interoperability-related  areas  that  are  addressed 
by  one  or  more  of  the  policy  documents  shown  in  the  rectangles  in  the  middle  of  the  diagram. 
This  list  of  topic  areas  was  generated  in  the  same  manner  as  described  in  the  previous  slide, 
by  reviewing  the  documents  and  synthesizing  the  collections  of  topic  areas  addressed  by  the 
documents.  These  topic  areas  are  spectrum,  training,  certification,  testing,  IA/CIP,  standards, 
acquisition,  GIG,  architectures,  NR-KPPs,  ISPs,  and  net-centric  checklists.  As  in  the  previous 
slide,  this  list  of  topic  areas  is  meant  to  convey  the  breadth  and  depth  of  the  discussions  related 
to  interoperability  in  the  policy  documents. 

The  collection  of  interoperability-related  policy  documents  included  in  our  analysis  is 
shown  in  the  rectangles  in  the  center  of  the  slide.  We  generated  this  collection  of  policy  docu¬ 
ments  by  starting  with  key  interoperability  documents,  including  DoDI  4630.8  (DoD,  2004g) 
and  CJCSI  6212. 01D  (JCS,  2006  [2007]).  We  built  the  collection  by  reviewing  all  of  the 
interoperability  documents  mentioned  in  the  initial  set  of  documents  and  supplemented  that 
list  with  other  publications  identified  by  DoD.  The  final  collection  is  shown  in  the  aggregation 
of  rectangles  in  the  middle  of  the  diagram  and  listed  in  Appendix  A.  The  rectangles  represent¬ 
ing  the  interoperability  policy  documents  are  colored  according  to  the  type  of  document  the 
rectangle  represents  using  the  color  scheme  shown  at  the  bottom  of  the  slide. 

As  in  the  previous  slide,  the  color-coordinated  lines  from  the  circles  in  the  bottom  row 
to  the  document  rectangles  in  the  center  indicate  that  the  linked  documents  address  the  topic 
areas  named  in  the  circles. 
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In  this  view,  the  ovals  at  the  top  of  the  slide  represent  the  primary  authors  of  the  docu¬ 
ments.  A  solid  black  line  from  an  oval  to  a  document  indicates  that  the  organization  named  in 
the  oval  is  a  primary  author  of  the  linked  document.  Some  policy  documents  show  multiple 
authors,  but  most  show  only  one  organization  as  the  author. 

While  this  particular  overview  of  DoD  interoperability-related  policy  documents  may 
not  be  optimal  for  tracing  the  origins  and  content  of  a  particular  document,  this  overview 
does  show  by  the  large  number  of  links  emanating  from  ASD(NII)  to  the  collection  of  policy 
documents  that  this  organization  (and  its  predecessor,  Assistant  Secretary  of  Defense  for  Com¬ 
mand,  Control,  Communication,  and  Intelligence  [ASD(C3I)]  has  generated  the  most  policy 
statements  regarding  interoperability.  Likewise,  the  JCS  have  issued  many  interoperability- 
related  policies.  The  documents  that  address  the  widest  range  of  interoperability  topic  areas 
can  be  identified  by  observing  the  bottom  portion  of  this  overview.  Since  DoDI  4630.8  and 
CJCSI  621 2. 01 D  have  the  most  lines  linking  them  to  topic  areas,  it  is  evident  that  these  two 
documents  address  the  most  topic  areas  and  thus  are  policy  documents  central  to  interoper¬ 
ability.  This  fact,  plus  the  observation  that  both  DoDI  4360.8  and  CJCSI  6212. 01D  contain 
interoperability  policy  statements  that  address  11  of  the  12  topic  areas  shown  in  circles  at  the 
bottom  of  the  slide  (spectrum,  training,  certification,  testing,  IA/CIP,  standards,  acquisition, 
GIG,  architectures,  NR-KPPs,  and  ISPs),  identify  these  documents  as  prime  candidates  for 
review  in  our  quest  to  uncover  inconsistencies  and  redundancies. 

In  addition,  by  observing  the  number  of  lines  emanating  from  each  topic  area  along  the 
bottom  row  of  circles,  one  can  identify  the  topic  areas  addressed  by  the  largest  number  of  policy 
documents.  This  observation  is  useful  because  it  gives  an  initial  set  of  areas  and  documents 
that  contain  potential  conflicts  in  policy  consistency  and  redundancy.  In  this  case,  topics  such 
as  testing,  IA/CIP,  standards,  acquisition,  GIG,  and  (to  a  slightly  lesser  extent)  architectures 
are  the  most-linked  topic  areas. 
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Using  a  presentation  scheme  similar  to  that  of  the  last  slide,  this  slide  shows  a  single  topic 
area  in  the  circle  at  the  bottom — legacy  systems.  The  policy  documents  are  indicated  by  col¬ 
ored  rectangles  in  the  center  of  the  slide.  The  colors  follow  the  scheme  of  previous  slides  as 
indicated  at  the  bottom  of  the  figure.  The  ovals  at  the  top  of  the  chart  show  the  agencies  that 
are  the  principal  authors  of  the  documents.  A  line  from  an  oval  to  a  document  indicates  that 
the  agency  named  in  the  oval  is  a  principal  author  of  the  linked  document.  Lines  emanating 
from  the  “Legacy  Systems”  circle  at  the  bottom  of  the  slide  to  documents  in  the  center  rectan¬ 
gles  indicate  that  the  linked  documents  include  policy  statements  about  how  DoD  addresses 
legacy  systems.  Policies  that  address  legacy  systems  are  indicated  separately  from  other  topics 
in  the  previous  slide  because  legacy  systems  have  been  a  significant  source  of  interoperabil¬ 
ity  problems.  We  can  see  that  CJCSM  3170.01  B  (JCS,  2005b);  GIG  capstone  requirements 
document  (CRD)  Joint  Requirements  Oversight  Council  Memorandum  (JROCM)  134-01 
(DoD,  2001);  CJCSI  6212.01D  (JCS,  2006  [2007]);  DoDD  8581. IE  (DoD,  2005a);  DoDI 

8551.1  (DoD,  2004J);  DoDI  5200.40  (DoD,  1997);  DoDD  8100.2  (DoD,  2004d);  DoDD 

8190.1  (DoD,  2000);  “DoD  Net-Centric  Data  Strategy”  (DoD  CIO,  2003);  Joint  Battle 
Management  Command  and  Control  (DoD,  2003a);  “End-to-End  IA  Component  of  the  GIG 
Integrated  Architecture”  (DoD,  2006c);  and  Military  Communications-Electronics  Board 
(MCEB)  publication  1  (MCEB,  2002)  all  contain  policies  regarding  legacy  systems. 
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Significant  Overiap  Exists  Between  DoDi 
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Of  the  many  policy  documents  on  the  preceding  charts,  the  two  overarching  documents  are 
DoD  instruction  4630.8  (DoD,  2004g),  issued  in  June  2004,  and  CJCSI  6212. 01D  (JCS, 
2006  [2007]),  issued  in  March  2006.  Though  both  documents  have  the  same  name,  the 
DoD  instruction  is  intended  to  provide  technical  requirement  guidance,  while  the  Joint  Staff 
instruction  is  intended  to  provide  interoperability  and  supportability  certification  requirement 
guidance. 

There  is  a  high  degree  of  overlap  in  what  these  two  documents  cover,  though  what  is  cov¬ 
ered  in  each  of  the  areas  is  not  necessarily  consistent.  Areas  of  overlap  include  the  NR-KPP,  the 
ISP,  verification  testing  and  certification  for  interoperability  and  supportability,  and  the  ICD, 
CDD,  and  CPD.  The  area  of  data  strategy  guidance  is  unique  to  CJCSI  6212. 01D,  and  the 
area  of  capability-focused  effects-based  IT/National  Security  Systems  (NSS)  interoperability 
process  overview  is  unique  to  DoDi  4630.8. 
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In  this  slide,  we  compare  the  guidance  offered  in  CJCSI  6212. 01D  (JCS,  2006  [2007]),  DoDI 
4630.8  (DoD,  2004g),  and  an  ASD(NII)  memorandum  (ASD[NII],  2005b). 

With  regard  to  the  ISP  process,  there  are  not  only  differences  between  CJCSI  6212. 01D 
(JCS,  2006  [2007])  and  DoDI  4630.8  (DoD,  2004g),  but  also  differences  between  each  of 
those  documents  and  the  2005  ASD(NII)  memo  (ASD[NII],  2005b),  the  subject  of  which  was 
ISP  acquisition  streamlining  pilot  program.  CJCSI  6212. 01D  addresses  tailored  ISP  (TISP), 
while  DoDI  4630.8  establishes  a  baseline  ISP  process,  and  the  ASD(NII)  memorandum  estab¬ 
lishes  and  encourages  an  alternative  process  called  the  ISP  pilot  program.  Some  of  the  dif¬ 
ferences  are  in  the  ISP  process  itself.  The  baseline  process  for  the  ISP  was  set  forth  in  DoDI 
4630.8.  That  process  was  significantly  streamlined  within  the  ISP  alternative  pilot  program  in 
the  ASD(NII)  memorandum.  Not  only  did  the  alternative  pilot  program  involve  significant 
streamlining,  but  the  use  of  that  alternative  was  also  strongly  encouraged  in  the  ASD  memo¬ 
randum.  One  subset  of  that  memorandum  addressed  a  TISP  applicable  to  acquisition  category 
(ACAT)  II  and  below  and  non-ACAT  programs.1  It  is  only  that  TISP  that  is  addressed  in 
CJCSI  6212. 01D,  not  the  remainder  of  the  streamlined  program.  Clearly,  a  PM  who  was  not 
familiar  with  the  details  of  each  policy  would  be  at  risk  of  violating  at  least  one  of  them.  At  the 
same  time,  even  if  PMs  were  aware  of  all  three,  simultaneous  compliance  with  the  intent  of  all 
three  policies  would  be  challenging  at  best.  Further  complicating  the  issue  is  that  there  is  no 
guidance  stipulating  when  one  policy  rather  than  another  should  be  followed. 


i 


See  DoD  (2006b)  for  more  details. 
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Turning  to  the  area  of  integrated  architectures,  we  find  that  there  are  significant  differ¬ 
ences  between  CJCSI  62f2.0fD  (JCS,  2006  [2007])  and  DoDI  4630.8  (DoD,  2004g)  regard¬ 
ing  what  integrated  architecture  products  are  needed  for  the  NR-KPP,  what  the  criteria  are  for 
an  interface  to  be  specified  as  a  key  interface,  and  what  is  required  as  part  of  a  key  interface  pro¬ 
file  (KIP).  First,  in  CJCSI  6212. 01D,  the  requirements  for  integrated  architecture  products  for 
the  NR-KPP  include  an  operational-view  (OV)-7  (logical  data  model),  system-view  (SV)-2 
(system  communication  description),  SV-11  (physical  schema),  and  technical  standards-view 
(TV)-2  (technical  standards  forecast).  None  of  these  is  required  in  DoDI  4630.8.  Second,  in 
CJCSI  6212. 01D,  the  specifications  of  criteria  for  an  interface  to  be  considered  a  key  interface 
include  (1)  whether  the  interface  has  a  very  large  number  of  point-to-point  interfaces  and  (2) 
whether  the  interface  includes  a  large  number  of  providers  or  consumers.  These  criteria  are 
not  included  as  conditions  for  designation  as  a  key  interface  in  DoDI  4630.8.  Third,  CJCSI 
621 2.01  D  requires  that,  as  part  of  a  KIP,  an  OV  and  SV  product  be  included  and  that  TV-1, 
TV-SV  bridge,  and  the  TV-2  and  procedures  for  testing  be  included.  In  DoDI  4630.8,  the 
refined  OV  and  SV  products  are  again  included,  but  there  are  also  requirements  for  an  ICD 
or  specification  for  the  engineering  management  plan;  configuration  management  plan;  and, 
again,  as  in  CJCSI  6212. 01D,  the  TV-1  and  TV-SV  bridge.  Procedures  for  testing  are  included 
as  a  requirement  in  DoDI  4630.8,  as  they  are  in  CJCSI  6212. 01D.  These  three  detailed  exam¬ 
ples  show  that,  in  the  integrated  architecture  area,  a  PM  is  again  challenged  to  comply  with 
all  of  the  stated  policies,  and,  again,  no  guidance  is  provided  to  indicate  which  policies  take 
precedence  over  others  when  all  cannot  be  satisfied. 
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Payoff  for  Combining  Guidance  is  Significant, 
but  implementing  it  Will  Pose  Challenges 


Advantages 

•Improved  policy  consistency 
•Reduced  volume  of  guidance 
PMs  have  to  review 

•Single  review  for 
combined  document  will 
lead  to  higher-quality  policy 

•More  effective  management  of 
programs  (requirements  and 
acquisition) 


Disadvantages 

•Current  division  reflects 
division  of  responsibilities 
and  authorities  between 
JCS  and  ASD(NII) 

•Combining  management 
process  will  be  a 
challenge 


RAND 


We  anticipate  that  significant  payoffs  will  accrue  from  combining  the  CJCSI  6212. 01D  (JCS, 
2006  [2007])  and  DoDI  4630.8  (DoD,  2004g)  guidance  into  a  single,  coordinated  guid¬ 
ance  document  but  realize  that  these  advantages  will  also  generate  two  significant  challenges. 
Advantages  that  we  anticipate  would  be  first,  and  perhaps  foremost,  improved  consistency 
between  policies  through  the  elimination  of  some  of  the  discrepancies  that  we  illustrated  in  the 
preceding  slide.  We  anticipate  that  a  single  review  for  a  combined  document  would  lead  to  a 
more  focused  and  in-depth  review  that  would  ultimately  lead  to  a  higher-quality  policy  guid¬ 
ance  document.  Finally,  a  combination  of  the  two  guidance  documents  should  lead  to  more 
effective  management  of  programs  by  combining  under  one  cover  the  requirement  perspective 
of  CJCSI  documentation  and  the  acquisition  perspective  of  ASD(NII)  documentation. 

At  the  same  time,  we  recognize  that  there  are  potential  disadvantages  and  challenges  to 
combining  the  policy  guidance.  Certainly,  a  management  challenge  could  be  created  if  the 
two  policies  were  combined.  The  current  arrangement  reflects  the  division  of  responsibilities 
and  authorities  that  exists  between  the  JCS  and  ASD(NII).  The  combination  of  policy  docu¬ 
ments  could  be  accompanied  by  the  merger  of  existing  management  and  oversight  processes 
into  a  single  process.  However,  this  does  not  necessarily  have  to  be  the  case,  and  we  do  not 
recommend  this  alternative  here.  If  a  unified  process  were  created,  it  would  be  a  challenge 
to  accurately  reflect  and  respect  the  current  division  of  responsibilities  between  the  JCS  and 
ASD(NII)  in  a  single  process. 
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Summary  -  DoD  Interoperability  and 
Supportability  Policy  Guidance 

•  Findings 

-  Large  number  of  policies  place  a  heavy  burden  on  the  PM 

-  Interoperability  policy  issuance  increased  dramatically  in  past 
few  years 

-  Key  guidance  documents  have  been  updated  separately  and 
contain  significant  overlap  and  conflicts  in  multiple  areas 

-  Key  elements  for  policy  compliance  missing:  joint  architectures 

-  Limited  availability  of  GIG-KIPs 

•  Needed  for  NR-KPP 

-  Metrics  for  NR-KPP  not  developed  /  specified 

•  Needed  for  development  of  TEMPs 

•  Required  to  pass  Milestone  C 

•  Recommendations 

Combine  DoDI  4630.8  and  CJCSI  6212.01  D  into  a  single 
overarching  DoD  interoperability  policy  document 
Develop  quantitative  metrics  for  NR-KPP,  based  on  essential 
GIG  functions  (as  defined  in  the  NCIDs) 

Develop  GIG-KPPs  based  on  essential  GIG  functions  (as  defined 
in  the  NCIDs 

21 

we  have  found  that  there  is  a  large  number  of  policy  documents,  with  that 
number  increasing  dramatically  in  recent  years.  The  need  to  be  familiar  with  all  of  these  docu¬ 
ments  places  a  heavy  burden  on  PMs.  Key  documents  have  been  updated  separately  without 
coordination,  and  this  has  resulted  in  a  situation  in  which  there  are  significant  overlaps,  incon¬ 
sistencies,  and  conflicts  among  policies  contained  in  the  documents. 

Several  elements  needed  for  policy  compliance  are  either  missing  or  limited  in  availability. 
These  include  joint  architectures  and  GIG  KIPs,  both  of  which  are  needed  for  the  NR-KPP,  as 
well  as  metrics  for  the  NR-KPP  that  are  needed  for  both  the  development  of  test  and  evalua¬ 
tion  master  plans  (TEMPs)  and  are  required  to  pass  MS  C. 

Our  primary  recommendation  is  that  DoDI  4630.8  (DoD,  2004g)  and  CJCSI  6212. 01D 
(JCS,  2006  [2007])  be  combined  into  a  single,  overarching,  DoD  interoperability  policy  docu¬ 
ment.  We  also  recommend  development  of  quantitative  metrics  for  the  NR-KPP,  based  on 
essential  GIG  functions  as  established  and  defined  in  NCIDs  documentation.  Similarly,  we 
recommend  development  of  GIG  KPPs,  again  based  on  essential  GIG  functions  as  established 
and  defined  in  the  NCIDs  documents.  We  envision  the  GIG  KPPs  as  applying  initially  to  core 
GIG  programs  and  eventually  to  systems  that  must  interconnect  to  these  key  systems  when  the 
full  functionality  of  the  GIG  core  becomes  available. 

We  recognize  that  achieving  these  recommendations  will  be  challenging,  but  the  advan¬ 
tage  of  having  consistent  interoperability  policy  that  is  easily  accessible  and  clear  will  result  in 
more  efficient  program  management  and,  ultimately,  more  efficiently  developed  and  effective 
systems. 


RAND 

In  summary, 
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Net-Centric  Interface  Documents 


Outline 


•  JCIDS 

•  Acquisition 

•  Interoperability 

•  NCIDs 

•  IA 

•  Summary 


RAND 


We  now  turn  our  attention  to  a  new  element  of  DoD  interoperability  policy  that  specifically 
addresses  GIG  component  system  integration  and  interoperability  policy  issues:  the  NCIDs. 


35 


36  Navy/OSD  Collaborative  Review  of  Acquisition  Policy  for  DoD  C3I  and  Weapon  Programs 


NCIDs  Requirement  Framework 
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This  slide  shows  the  NCID  requirement  framework  contained  in  the  draft  version  2  of  the 
NCIDs.  The  framework  has  three  tiers,  with  the  NCID000  GIG  NCID  serving  as  the  over¬ 
arching  policy  statement. 

The  NCIDs  are  the  product  of  several  working  groups  (WGs)  of  the  GIG  E2E  system 
engineering  (SE)  activity.  These  DoD-wide  working  groups  were  designed  to  have  representa¬ 
tion  from  services,  agencies,  GIG-related  acquisition  programs,  the  Office  of  the  Secretary  of 
Defense,  the  JCS,  and  the  intelligence  community,  although  some  working  groups  did  not 
have  representatives  from  some  DoD  organizations. 

The  objective  of  the  NCIDs  are  to  provide  the  minimum  set  of  functional  performance 
requirements  necessary  for  users’  applications  to  work  effectively  in  an  E2E  fashion  across  the 
GIG.  These  requirement  documents  cover  all  necessary  functional  components  of  the  GIG: 
transport,  applications,  IA,  computing,  and  services. 

Another  key  element  of  the  NCIDs  is  the  performance  allocation  (PA)  framework,  which 
is  discussed  later  in  this  chapter. 
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NCIDs  -  Purpose  and  Scope 

•  Purpose 

-  Enterprisewide  implementation  guidance  on  system 
interactions  across  the  GIG 

-  Specify  GIG  segment  functions  necessary  to  achieve 
desired  end-to-end  performance 

•  Provides  more  detailed  design  guidance  than  high-level 
architecture  products  (e.g.,  NCOW  reference  model) 

•  Scope 

-  Entire  GIG  SoS,  but  initially  focused  on  core  GIG 
programs 

•  Once  approved  NCIDs  to  become  mandatory  parts  of  the  DISR 

-  All  future  GIG  programs 

-  All  current  programs  of  record  that  are  pre-milestone  B 
within  60  days  of  April  25,  2005 

-  Legacy  programs  not  classified  “end  of  life”  to  develop 
modification  plan  to  reach  compliance  or  submit  request 
for  waiver  based  on  cost,  schedule,  or  performance 

RAND  impact 


The  purpose  of  the  NCID  is  to  establish  enterprisewide  implementation  guidance  on  system 
interactions  across  the  GIG  by  specifying  GIG  segment  functions  necessary  to  achieve  desired 
E2E  performance.  NCIDs  provide  more  detailed  design  guidance  than  do  high-level  archi¬ 
tecture  products  such  as  the  net-centric  operations  and  warfare  (NCOW)  reference  model. 
NCIDs  will  eventually  address  the  entire  GIG  system  of  systems,  but,  initially,  NCIDs  will 
focus  on  the  core  GIG  programs.  The  NCID  framework  has  not  yet  been  approved,  but,  once 
it  attains  approval  status,  NCIDs  will  become  mandatory  parts  of  the  Defense  Information 
Standards  Registry  (DISR). 

As  such,  when  they  are  approved,  it  is  envisioned  that  NCIDs  will  apply  to  all  future 
GIG  programs,  all  current  programs  of  record  that  were  pre-MS  B  on  June  24,  2005,  and 
all  legacy  programs  not  categorized  as  “end  of  life.”  Legacy  programs  that  are  not  categorized 
as  “end  of  life”  will  be  required  to  develop  modification  plans  to  reach  compliance  with  the 
NCID  framework  or  submit  a  waiver  based  on  disadvantageous  cost,  schedule,  or  performance 
impacts  imposed  by  NCID  compliance. 
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Technology  Standards  in  the  NCIDS  Are  Evolving,  Making  It 
Difficult  to  Provide  Meaningful  Guidance  to  PMs 

•  Some  implied  NCID  requirements  rely  on  nonexistent 
standards 

-  PMs  are  to  comply  with  applicable  DoD-approved  data  models  - 
very  few  exist 

-  QoS  signaling  (most  implementations  are  proprietary  today) 

•  Challenge  is  to  be  specific  to  enable  interoperability  and  broad 
to  leave  room  for  important  new  technologies  and  solutions 

-  “Chicken  or  egg”  issue  for  emerging  technologies 

-  In  many  areas  commercial  industry  leads  in  tech  development, 
and  standards  are  proprietary 

-  Suggested  standards  do  not  provide  sufficient  guidance  to  PMs 

•  Assumes  applications  will  be  services  on  high-speed  IP 
networks  -  not  accurate  for  all  tactical  users  for  many  years 

•  Unclear  as  to  what  GIG  core  services  will  deliver  and  what  PMs 
will  need  to  provide 

-  GIG  boundary  not  well  defined 


RAND 


The  technology  standards  in  the  NCIDs  are  currently  evolving.  This  implies  that  NCIDs, 
at  least  in  some  cases,  cannot  currently  provide  meaningful  and  specific  guidance  to  PMs 
on  which  technical  approaches  or  standards  they  should  follow.  Furthermore,  some  implied 
NCID  requirements  rely  on  standards  that  are  not  specified.  PMs  are  required  by  NCIDs 
to  comply  with  applicable  DoD-approved  data  models,  but  very  few  data  models  exist.  The 
current  NCID  requirement  framework  does  not  even  explicitly  include  data  as  a  segment.  In 
addition,  some  implementations  of  higher-level  GIG  functions,  such  as  QoS  signaling,  exist, 
but  many  implementations  are  currently  proprietary.  This  can  make  unified  compliance  with 
a  universal  GIG  QoS  standard  very  difficult  to  achieve  in  a  multivendor  network  such  as  the 
GIG. 

The  evolution  of  the  NCIDs  is  governed  by  two  principles  that  can  pose  conflicting  chal¬ 
lenges.  First,  NCIDs  must  be  specific  enough  to  enable  interoperability.  Second,  and  at  the 
same  time,  NCIDs  must  be  broad  enough  to  allow  for  incorporation  of  important  new  tech¬ 
nologies  and  solutions.  This  situation  creates  a  chicken-or-egg-first  issue  for  emerging  technolo¬ 
gies.  Further  complicating  the  picture  are  the  facts  that,  in  many  areas,  commercial  industry 
leads  in  technology  development  and  that  the  standards  that  commercial  industry  develops  are 
frequently  proprietary  and  may  not  easily  adapt  to  widespread  use  across  the  GIG.  In  addition, 
the  current  NCID  guidance  is  not  time  phased,  and  the  performance  assessment  framework 
is  not  tied  explicitly  to  operational  and  system  architectures.  Because  of  these  issues,  the  stan- 
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dards  currently  contained  in  the  NCIDs  (i.e.,  draft  version  2  of  the  NCIDs)  do  not  provide 
sufficient  guidance  to  PMs  to  achieve  interoperability.1 

Therefore,  the  current  NCID  requires  further  development  before  it  can  be  considered 
workable  guidance.  At  a  minimum,  some  assumptions  need  to  be  closely  examined  for  accu¬ 
racy,  and  a  more  detailed  definition  of  GIG  core  services  should  be  provided.  For  example,  in 
its  current  state,  the  NCID  framework  assumes  that  applications  will  be  services  with  high¬ 
speed,  internet  protocol  (IP)-based  networks.  This  assumption  will  not  be  generally  accurate 
for  all  tactical  users  and  especially  not  for  tactical  users  on  the  move.  The  current  NCID  is  also 
unclear  about  what  GIG  core  services  will  deliver  and  what  PMs  will  need  to  provide.  Without 
clear  and  specific  definition  of  where  GIG  core  enterprise  service  functionality  and  reach  end 
and  where  local  program  services  being  developed  by  tactical  PMs  begin,  PMs  will  have  insuf¬ 
ficient  guidance  on  how  to  design  systems  that  will  be  interoperable  with  the  GIG. 


i 


In  this  case,  RAND’s  findings  have  been  confirmed  by  comments  forwarded  to  RDA  CHSENG  by  some  Navy  PMs. 
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Estimating  E2E  GIG  System  Performance:  NCID 
Performance  Allocation  Across  Segments 

•  GIG  segments  to  be  interoperable 

-  in  accordance  with  approved  requirements  documents 

-  will  also  adhere  to  PAs  defined  by  NCIDs 

•  PA  framework  will  provide  overall  PA  specifications 

-  PA  specifications  will  be  used  by  ASD(NII)  GIG  SE  to  assess 
E2E  performance  of  the  GIG  in  specific  mission  areas 

•  Findings 

-  Most  version  2.0  NCIDs  did  not  include  PA  specifications,  to  be 
included  in  3.0 

-  Improved  net-centric  IER  data  sets  and  server  architectures 
needed 

•  Data  centers,  DNS,  BFT/COP/UDOP  data  streaming 

•  Appropriate  JCIDS  CBA  results 

-  Additional  high-level  simulation  tools  may  be  needed  to  assess 
network  performance  and  scalability 

-  GIG  TEN  NCIDs  will  be  key  for  many  PMs 

RAND 


The  NCIDs  address  PA  across  segments,  stipulating  that  GIG  segments  be  interoperable  in 
accordance  with  approved  requirement  documents.  The  GIG  segments  will  also  be  required 
to  adhere  to  PAs  defined  by  NCIDs.  The  PA  framework  will  provide  overall  PA  specifications. 
These  PA  specifications  will  be  used  by  the  ASD(NII)  GIG  SE  to  assess  E2E  performance  of 
the  GIG  in  specific  mission  areas. 

Our  review  of  the  version  2.0  NCIDs  shows  that  most  of  the  documents  did  not  include 
PA  specifications,  but  these  are  expected  to  be  included  in  version  3.0  NCIDs.  We  also  found 
that  improved  net-centric  information  exchange  requirement  (IER)  data  sets  and  server  archi¬ 
tectures  will  be  needed,  particularly  for  data  centers,  domain  name  services  (DNSs),  and  blue 
force  tracking  (BFT),  and  for  common  operational  picture  (COP)  and  user-defined  opera¬ 
tional  picture  (UDOP)  data  streaming.  Results  of  appropriate  JCIDS  CBAs  could  be  used.  We 
also  found  that  additional,  high-level  simulation  tools  may  be  needed  to  assess  network  perfor¬ 
mance  and  scalability.  Finally,  we  believe  that  the  GIG  tactical  edge  network  (TEN)  NCIDs 
will  be  a  key  policy  resource  for  many  PMs. 
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We  now  turn  our  attention  to  IA.  We  uncovered  38  documents  that  contain  policy  statements 
that  address  I  A.  The  review  of  all  38  is  discussed  in  this  chapter. 
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Global  Map  of  DoD  lA-Related  Policy  Documents 


Using  the  same  structure  we  employed  earlier  to  view  policy  documentation  relating  to  interop¬ 
erability,  this  slide  shows  a  mapping  of  DoD  IA-related  policy  documents  to  their  respective 
authors  and  the  different  IA  topic  areas  that  the  policies  in  the  documentation  cover. 

In  this  slide,  the  ovals  along  the  top  row  represent  the  entity  author  of  the  IA  document. 
These  entity  authors  include  the  Director  of  Central  Intelligence  (DCI),  JCS,  ASD(NII),  NSA, 
USD(AT&L),  Congress,  and  the  Committee  on  National  Security  Systems  (CNSS).  The  doc¬ 
uments  represented  by  colored  rectangles  in  the  middle  of  the  slide  are  the  DoD  policy  docu¬ 
ments  and  other  government  policy  documents  related  to  the  issue  of  IA.  The  color  of  the 
rectangle  associated  with  each  document  indicates  the  type  of  document  per  the  legend  shown 
at  the  bottom  of  the  slide.  A  solid  black  line  between  an  oval  and  a  rectangle  indicates  that  the 
entity  named  in  the  oval  is  a  primary  author  of  the  document  represented  by  the  rectangle. 

The  circles  near  the  bottom  of  the  slide  are  the  IA  topic  areas  addressed  by  the  policy 
documents.  These  topic  areas  are  interoperability;  training;  secret  compartmentalized  infor¬ 
mation  (SCI);  computer  network  defense  (CND);  port,  protocol,  and  service  management 
(PPSM);  public  key  infrastructure  (PKI);  baseline  of  IA  controls;  the  DITSCAP  process;  GIG; 
and  acquisition,  common  criteria,  wireless,  space,  international  networks,  and  IPv6.  The  slide 
shows  each  topic  area  in  a  different  color,  and  color-coordinated  lines  emanating  between  a 
circle  and  a  document  indicate  that  the  document  addresses  the  linked  topic  area  in  its  policy 
statements. 

This  overview  diagram  shows  that  most  of  the  policies  relating  to  IA  are  authored  by 
ASD(NII)  (and  its  predecessor,  ASD[C3I]).  By  observing  the  number  of  lines  between  each 
circled  topic  area  and  the  rectangles,  one  can  conclude  that  many  policy  documents  address 
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topic  areas  such  as  baseline  IA  controls,  the  DITSCAP  process,  the  GIG,  and  acquisition.  The 
GIG  is  the  most  highly  impacted  area,  with  13  documents  addressing  GIG  policy.  Similarly, 
by  observing  the  number  of  lines  emanating  from  each  document  rectangle,  one  can  conclude 
that  the  policy  documents  that  address  the  most  IA  topic  areas  are  DoDD  8500.1  (DoD, 
2002d)  and  DoDI  8500.2  (DoD,  2003b).  These  two  documents  are  outlined  in  red  to  indicate 
that  they  are  key  IA  policies. 
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Issuance  of  DoD  lA-Related  Policy  Documents 


Reviewed  over  38  documents: 

-  19  Define  or  establish  policy 

-  14  Guidance  or  instructions 

-  17  Acquisition-related  guidance  or 
instructions 

-  17  GIG-related  policies  and  procedures 

-  2  IPv6-related  policies  and  procedures 


[Conn^.^ec  Act|  .  |ssueS  Q1  1988 


|U"naie996Qnen|  -  ISSUeS  Q1  1996 


*  Estimated  time  frame  for  issuance 
**  Canceled  on  issuance  of  CJCSI  6212.01  D 
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DoD  directives  &  instructions 
JCS  documents 
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3  Non-DoD  or  gov't 
I  DoD  agency  doc 


This  slide  shows  the  currency  and  multiplicity  of  DoD  policy  documents  that  address  I  A. 

A  timeline  by  quarter,  starting  in  1997  and  ending  in  2006,  is  shown  at  the  bottom 
of  the  slide.  Policy  documents  related  to  IA  are  shown  in  labeled  rectangles  above  the  time¬ 
line  according  the  release  date  of  the  document.  The  type  of  document  is  also  shown  by  the 
colors  of  the  rectangles.  Green  indicates  that  the  document  is  a  DoD  directive  or  instruction. 
Magenta  represents  JCS  documents.  Yellow  represents  a  DoD  CIO  memorandum.  Purple 
indicates  NCIDs.  Gold  indicates  a  non-DoD  government  document.  Finally,  orange  indicates 
a  DoD  agency  document.  Note  that  this  color  scheme  differs  from  the  one  presented  earlier  for 
interoperability  documents. 

This  slide  shows  that  there  is  a  dramatic  increase  in  the  number  of  policy  documents 
related  to  I A  that  have  been  issued  since  the  first  quarter  of  2003,  with  nine  policy  statements 
issued  in  2003,  followed  by  another  nine  in  2004,  and  eight  more  in  2005.  TTie  timeline  shows 
some  promise  of  a  decreased  rate  in  2006  or  2007.  For  example,  this  timeline  shows  only  one 
new  policy  document  for  IA  and  information  security  certifications  being  issued  in  2006, 
DoDI  8510.bb,  Defense  Information  Assurance  Certification  Accreditation  Program  (DIACAP) 
(DoD,  2006a).  However,  there  are  reasons  to  believe  that  IA  policy  issuance  will  again  increase 
after  DIACAP  is  implemented. 


Information  Assurance  Policy  45 


Challenges  of  Applying  DoD  I  A  Policies  to  DoD 

Information  Systems 

•  Large  number  of  IA  policy  documents 

-  Did  not  find  significant  inconsistencies  in  DoD  IA  policy 

-  Identified  some  inconsistencies  with  DoD  and  Navy  IA  policies 

-  New  policy  now  in  effect:  DoDI  8510.bb  DIACAP 

•  DIACAP  8510.bb  released  as  interim  guidance  in  July  2006,  a  significant 
number  of  documents  will  have  to  be  updated  to  maintain  consistency 

•  8500  Series  includes  exemptions  for  embedded  systems  and  platform  IT 
not  connected  to  the  GIG,  but  an  increasing  number  of  exempted 
systems  will  be  connected  in  real  time  to  the  GIG,  including 

•  weapon  systems  ->  UAVs/UGVs 

•  voice  communications  ->  voice  over  IP 

•  embedded  systems  ->  radar  (e.g.,  Navy  C EC) 

•  Should  a  universal  IA  policy  be  created  and  applied  to  all  DoD 
warfighting  systems  or  development  of  separate  IA  policies  for 
previously  exempted  systems? 

•  Changes  in  the  information  infrastructure  are  making  new  demands  on  IA 

-  Traditional  enclave  and  system  boundaries  are  breaking  down 

RAND  30 


This  slide  summarizes  the  major  findings  regarding  DoD  IA  policy. 

As  shown  in  the  previous  slide,  a  large  number  of  different  policies  and  continuing  poli¬ 
cies  is  being  issued  by  DoD,  the  military  services,  and  DoD  agencies  concerning  IA.  This  fact 
makes  it  difficult  to  keep  this  body  of  policy  statements  synchronized  and  up  to  date.  However, 
as  opposed  to  the  case  with  interoperability,  the  body  of  DoD  IA  policy  appears  to  be  inter¬ 
nally  consistent.  Our  review  reveals  only  some  minor  inconsistencies  between  DoD  IA  policies 
and  Navy  IA  policies.1 

A  major  update  to  DoD  IA  policy,  DoDI  8510. bb  (DoD,  2006a)  was  released  as  interim 
guidance  in  July  2006.  DIACAP  replaces  DITSCAP.  The  IA  topic-area  chart  presented  earlier 
shows  that  seven  other  IA  documents  address  DITSCAP,  so  those  seven  documents  and  pos¬ 
sibly  others  will  have  to  be  updated  to  maintain  the  internal  DoD  IA  policy  consistency  with 
the  new  DIACAP  policy. 


1  It  should  be  noted  that  only  minor  inconsistencies  between  DoD  and  Department  of  the  Navy  IA  policies  were  identified. 
Specifically,  DoD  instruction  5200.40  (DoD,  1997,  §2.3)  states  that  certification  and  accreditation  (C&A)  applies  to  any 
IT  system,  while  SECNAV  instruction  5239.3A  (SECNAV,  2004,  p.  12,  §7.e)  states  that  C&A  is  required  only  for  systems 
that  are  connected  to  GIG.  In  addition,  DoD  directive  8500.1  (DoD,  2002d,  p.  2,  §§2.1. 2.1,  2. 1.2. 6)  and  DoD  instruc¬ 
tion  5200.40  state  that  the  scope  of  the  IA  policies  would  cover  all  NSS  IT  systems  “including  Special  Access  Program”  and 
“stand  alone  information  systems.”  SECNAV  5239.3A  (p.  5,  §5.b)  states  that  its  policies  will  not  superscede  DCI  policies 
regarding  SCI  or  special  access  programs.  Finally,  DoD  instruction  5200.40  (p.  14,  §E3.3.3.4)  defines  the  user  representa¬ 
tive  as  the  liaison  for  the  user  community  to  the  DITSCAP  process.  Neither  OPNAV  instruction  5239. IB  (OPNAV,  1999) 
nor  SECNAV  instruction  5239.3A  provides  guidance  for  identifying  a  user  representative  for  the  DITSCAP  process. 
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Another  important  finding  that  was  discovered  during  this  investigation  was  the  chal¬ 
lenges  associated  with  the  exceptions  built  into  the  DoD  8500-series  documents.  These  excep¬ 
tions  exclude  certain  types  of  IT  systems  from  the  requirements  for  IA.  These  exceptions 
include  information  technologies  embedded  into  systems  such  as  weapon  platforms  as  well  as 
information  technologies  that  were  considered  not  to  be  in  any  way  connected  to  the  GIG. 
However,  we  are  seeing  over  time  that  new  technologies  are  being  introduced  into  some  systems 
that  were  originally  exempt  and  that  these  new  technologies  are  challenging  the  preconceived 
notion  of  what  should  be  exempt  from  IA  requirements  or  policy.  For  example,  unmanned 
aerial  vehicles  (UAVs)  and  unmanned  ground  vehicles  (UGVs)  in  the  military  forces  were  orig¬ 
inally  expected  to  be  exempt  from  IA  requirements.  However,  UAVs  and  UGVs  in  the  military 
forces  have  evolved,  and  these  platforms  are  now  heavily  integrated  into  networks  and  into  IT, 
so  any  exemptions  that  might  allow  such  systems  to  escape  meeting  the  same  IA  requirements 
need  to  be  revisited. 

Similarly,  voice  communication  was  originally  considered  exempt  from  many  of  the 
requirements  for  IA  or  information  security.  Since  that  decision,  we  have  moved  to  increas¬ 
ing  use  of  voice  over  IP  in  some  systems.  This  development  warrants  a  reinvestigation  of  the 
exemption  of  voice  communication  from  IA  requirements. 

Even  certain  embedded  systems — such  as  radar,  which  used  to  be  considered  separate 
and  not  part  of  network  communication — now,  increasingly,  are  becoming  part  of  network 
communication.  Applications  such  as  the  Navy’s  cooperative  engagement  capability  (CEC), 
in  which  radar  information  is  shared  among  both  aircraft  and  surface  platforms  such  as  Navy 
ships  to  provide  a  more  complete  picture  of  radar  space,  is  an  example  of  an  embedded  system 
that  was  originally  considered  a  good  case  for  exemption  but  that  may  no  longer  meet  the 
intent  of  the  DoD  8500-series  exceptions. 

These  examples  illustrate  that  systems  that  were  once  considered  embedded  and  isolated 
to  the  platform  itself  are  now  sharing  information  throughout  the  GIG  to  provide  more  com¬ 
plete  views  of  the  COP  in  military  operations.  This  means  that  we  need  to  identify  the  exemp¬ 
tions  that  exist  for  those  systems  and  determine  whether  the  exemptions  should  still  apply  if 
the  system  is  now  sharing  information  over  the  military  networks. 

Our  analysis  of  the  exemption  issue  leads  to  more  questions:  Should  there,  in  fact,  be  a 
universal  IA  policy  that  applies  to  all  DoD  warfighting  IT  systems?  Is  such  a  universal  policy 
necessary,  or  is  the  current  policy  with  synchronization,  patches,  and  updating  sufficient  to 
reflect  changes  in  how  IT  is  being  used?  Which  option  will  best  be  able  to  provide  adequate 
policy  guidance  for  providing  IA? 

Finally,  changes  in  IT  and  the  military’s  increasing  use  of  IT  are  creating  new  demands 
on  current  IA  policy.  The  expected  increase  of  GIG  tactical  networks  and  the  future  vision  for 
GIG  tactical  networks  in  which  there  will  be  communities  of  interest  (COIs)  that  are  created 
dynamically  for  members  just  prior  to  a  mission  are  examples  of  areas  in  which  current  IA 
policy  may  fall  short.  Other  areas  include  IA  policy  for  the  products  of  sensor  and  informa¬ 
tion  fusion.  The  full  concept  of  COIs  is  still  being  reviewed,  but  the  use  of  IPv6  indicates  an 
increasing  use  of  dynamic  networks  and  ad  hoc  networks.  These  types  of  networks  create  a  tre¬ 
mendous  strain  and  challenge  for  traditional  concepts  of  enclaves  and  boundaries.  Therefore, 
new  policies  will  need  to  be  developed  that  will  be  able  to  accommodate  the  dynamic  networks 
that  may  emerge  with  thousands  of  nodes  for  a  single  mission. 

Another  challenge  on  the  horizon  comes  with  the  introduction  of  IPv6.  IPv6  will  provide 
a  large  number  of  IP  addresses,  and  this  provision  will  potentially  allow  a  single  soldier  to  be 


Information  Assurance  Policy  47 


equipped  with  several  devices,  each  of  which  has  its  own  IP  network,  its  own  IP  address,  and, 
possibly,  different  security  classifications  for  each  IP  address.  Therefore,  concepts  and  methods 
for  managing  a  multiplicity  of  IP  addresses,  each  with  its  own  set  of  access  and  use  characteris¬ 
tics,  will  be  have  to  be  developed.  The  management  system  developed  will  have  to  address  how 
to  achieve  IA  in  such  future  operating  environments. 


CHAPTER  SEVEN 

Summary 


Outline 


•  JCIDS 

•  Acquisition 

•  Interoperability 

•  NCIDs 

•  IA 

•  Summary 
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A  summary  of  our  study  follows. 
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Summary  -  Findings 

•  There  has  been  a  proliferation  of  policy  documents 
in  recent  years  and  guidance  is  often  not  actionable 

-  policy  issuance  has  sharply  increased  in  the  past  few 
years 

-  we  found  conflicts  and  overlaps  in  interoperability  policy 

a  GIG  guidance  and  standards  are  still  evolving  and 
not  yet  “stable” 

-  rapid  technology  change 

-  PA  framework  under  development 

-  key  technologies  being  developed  under  the  leadership  of 
commercial  industry 

•  Large  number  of  IA  policies 

-  many  will  have  to  be  updated  to  be  consistent  with  DoDI 
8510.bb,  which  is  now  in  effect 

RAND  32 

In  this  briefing,  we  have  examined  a  broad  set  of  DoD  IT-  and  interoperability-related  poli¬ 
cies.  We  found  that  policy  issuance  has  increased  sharply  in  recent  years.  By  mapping  the 
policies  to  key  areas  and  examining  the  details,  we  found  that  some  conflicts  and  redundan¬ 
cies  exist  among  the  collection  of  policies  included  in  our  study.  Most  of  these  conflicts  are 
in  the  area  of  interoperability  policy.  The  conflicts  and  redundancies  contribute  to  the  inac- 
tionable  nature  of  some  of  the  policies. 

We  found  GIG  guidance  and  standards  to  be  imprecise  from  a  technical  standpoint 
and  still  evolving.  The  changes  are  being  fueled  by  rapid  changes  in  technology.  In  addition, 
DoD  standards  development  is  difficult  because  industry  leadership  in  standards  development 
imparts  a  proprietary  nature  on  some  resulting  standards. 
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Summary  -  Recommendations 

•  Streamline  DoD  IT  policy  and  make  it  more  actionable  by  PMs 

-  Reduce  number  of  interoperability  and  IA  policies 

-  GIG  TEN  NCIDs  will  be  key  for  many  PMs 

•  In  JCIDS,  improve  requirements  traceability  from  COCOM  needs 
statements  to  program  development  documents 

•  For  interoperability,  DoD  should  combine  CJCSI  6212.01D  and  DoDI 
4630 

-  Develop  under  the  joint  leadership  of  ASD(NII)  and  J6 

•  Acquisition 

-  Establish  technology  risk  areas  and  TRLs  for  GIG  interoperability  areas 

-  Ensure  independent  tech  assessment  of  GIG  program  interoperability 
approaches  by  appropriate  SYSCOMs  or  DISA 

-  Move  Milestone  B  to  PDR,  at  least  for  high-IT  content  programs 

•  In  version  3.0  NCIDs 

-  Clearly  define  GIG  core  services,  which  programs  will  provide  them, 
needed  network  bandwidth  -  to  reduce  potential  duplication  of  effort 

-  Reduce  reliance  on  proprietary  standards  for  functions  that  must  cross 
segment  boundaries  -  QoS  signaling 

-  Service  operational  communities  should  review  GIG  use  cases  to  be 
used  in  the  PA  framework 


Our  examination  of  DoD  IT-related  policy  leads  to  several  recommendations.  First,  we  recom¬ 
mend  that  DoD  IT-related  policy  be  streamlined  by  reducing  the  number  of  policies  in  spe¬ 
cific  areas,  such  as  interoperability  and  IA.  This  move  will  allow  PMs  and  other  stakeholders 
to  determine  which  policies  apply  to  their  particular  programs  and  what  party  is  responsible 
for  the  attendant  actions,  events,  and  deliverables.  Along  this  same  vein,  the  policies  should 
be  more  actionable  for  PMs.  This  last  point  is  especially  important  for  the  GIG  TEN  NCIDs, 
which  will  be  a  key  policy  resource  for  many  DoD  managers  of  tactical  programs. 

Second,  regarding  JCIDS,  the  requirement  traceability  from  COCOM  needs  statements 
to  program  development  documents  should  be  improved,  especially  as  these  pertain  to  interop¬ 
erability  requirements.  Third,  interoperability  policy  would  be  improved  by  combining  CJCSI 
6212. ID  (JCS,  2006  [2007])  with  DoDI  4630  (DoD,  2004g)  and  developing  the  new  com¬ 
bined  policy  under  the  joint  leadership  of  ASD(NII)  and  the  command,  control,  communica¬ 
tion,  and  computer  (C4)  directorate  (J6).  Implementation  of  these  first  three  recommendations 
would  reduce  the  volume  of  literature  that  addresses  interoperability  and  IA  policy  and  make 
the  policy  more  user  oriented. 

Fourth,  with  regard  to  acquisition,  technology  risk  areas  and  TRLs  for  GIG  functional 
areas  should  be  established.  In  addition,  the  acquisition  policy  should  ensure  independent 
assessment  of  technology  risk  for  programs  by  appropriate  SYSCOMs  or  DISA.  TIigh-IT  con¬ 
tent  programs  will  have  a  higher  probability  of  meeting  cost,  schedule,  and  performance  goals 
if  MS  B  is  moved  to  the  current  PDR  point  on  the  acquisition  timeline.  Implementing  this  set 
of  recommendations  will  result  in  a  better  understanding  of  technology  risk  and  how  best  to 
handle  it. 
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We  also  recommend  that  version  3.0  NCIDs  clearly  define  GIG  core  services,  identify 
which  programs  will  provide  them,  specify  the  network  bandwidth  needed  to  support  them, 
and  state  what  functionality  they  will  provide.  This  clarification  will  reduce  the  potential  for 
duplication  of  effort  by  the  PMs  and  will  aid  in  the  convergence  of  interface  specifications. 
Furthermore,  version  3.0  NCIDs  should  reduce  the  reliance  on  proprietary  standards  for  func¬ 
tions  that  must  cross  segment  boundaries,  such  as  QoS  signaling.  In  addition,  version  3.0 
NCIDs  should  allow  service  operational  communities  not  only  to  review  GIG  use  cases  that 
are  to  be  used  in  the  performance  allocation  framework,  but  also  to  be  offered  greater  oppor¬ 
tunity  to  participate  in  their  development. 

Implementation  of  our  recommendations  is  likely  to  present  a  number  of  challenges,  but 
the  potential  benefits  of  implementing  these  recommendations  to  DoD  IT-related  policy  far 
outweigh  the  obstacles  that  may  have  to  be  overcome  otherwise. 
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Additional  Details  for  Selected  Topics 


DSB  and  DAPA  Recommendations  to  Improve 
Acquisition  System  Performance 

•  DSB 

-  Move  from  requirement-based  to  judgment-based  execution 

•  Force  capability  trade-offs  to  maintain  cost  and  schedule 

-  Use  technical  red  teams  to  independently  assess  tech  feasibility 

-  Rigorously  enforce  the  TRL  process 

-  Include  integration  risk  and  manufacturing  readiness  in  the 
technical  assessment. 

•  DAPA 

-  Shift  to  time-certain-development  -  make  schedule  a  KPP 

•  Require  IOC  to  occur  no  later  than  six  years  after  Milestone  A 

•  T rade  off  technical  performance  to  achieve  schedule 

-  Establish  TRLs  for  system  design 

-  Realign  Milestone  B  to  occur  at  preliminary  design  review 

-  Require  program  TEMPs  and  initial  OT&E  plans  to  be  completed 
prior  to  Milestone  B 

-  Enhance  PM  authority  so  they  can  defer  non-KPP  requirements  to 
later  program  blocks  or  upgrades 

RAND 


This  appendix  contains  a  few  additional  slides  that  further  illuminate  points  made  in  the  body 
of  this  document. 

Our  study  cited  several  recommendations  from  the  DSB  and  DAPA  to  improve  acquisi¬ 
tion  system  performance.  The  DSB  recommended  that  DoD  move  from  requirement-based  to 
judgment-based  execution  and  make  force  capability  trade-offs  to  maintain  cost  and  schedule 
goals.  The  DSB  also  recommended  that  DoD  use  technical  red  teams  to  independently  assess 
technological  feasibility  and  rigorously  enforce  the  TRL  process  and  include  integration  risk 
and  manufacturing  readiness  in  the  technical  assessment. 

DAPA  recommended  a  shift  to  time-certain  development  and  making  the  schedule  a  key 
performance  parameter  (KPP).  In  addition,  it  recommended  that  there  be  a  requirement  for 
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initial  operational  capability  (IOC)  to  occur  no  later  than  six  years  after  MS  A  and  that  trade¬ 
offs  in  technical  performance  be  made  to  achieve  schedule  goals.  It  also  recommended  that 
TRLs  be  established  for  system  design;  that  MS  B  be  realigned  at  PDR;  that  program  TEMPs 
and  initial  operational  test  and  evaluation  (OT&E)  plans  be  completed  to  pass  MS  B;  and, 
finally,  that  PM  authority  be  enhanced  so  that  PMs  can  defer  non-KPP  requirements  to  later 
program  blocks  or  upgrades. 
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DIACAP  Provides  a  New  Certification  and 
Accreditation  Process 

•  DIACAP  establishes  new  procedures  for  authorizing  the  operation  of  DoD  information 
systems  (ISs) 

-  Will  replace  DoD  5200.40  (DoD,  1999)  (DITSCAP)  and  DoD  6-8510  (DoD  CIO, 
2000a)  (DITSCAP  manual) 

-  Comply  with  DoDI  8500.2  (DoD,  2003b),  legislative  (FISMA),  and  1C  (DCID  6/1) 
(Director  of  Central  Intelligence,  1995  [2003])  policies. 

-  Shift  from  individual  systems  to  enterprise  perspective  consistent  with  the  GIG 

•  IA  controls  based  on  mission  assurance  category  (MAC)  and  confidentiality  level  (CL) 

•  Responsibility  for  establishing,  managing,  and  tracking  the  status  of  DoD  IS  within 
the  DIACAP  rests  with  the  DoD  component  IA  programs* 

•  DIACAP  includes  online  portal  and  Web  tools  for  supporting  DIACAP  execution 

•  Findings: 

-  Reaccreditation  every  year  instead  of  every  three  years 

-  All  ISs  inherit  enterprise  IA  standards  and  requirements  as  baseline 
requirements;  however 

•  Certification  and  accreditation  is  still  structured  on  a  per-system  basis 

•  System  IA  controls  can  be  augmented  to  address  local  threats  or  vulnerabilities 

•  CA  process  suited  for  static  environments;  not  well  suited  for  dynamic  COIs 

•  What  defines  the  “enterprise”?  Who  defines  the  enterprise  IA  baseline  controls? 

DA  Kin  *state"<2°°5> 
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This  slide  outlines  the  new  certification  and  accreditation  process  that  DIACAP  will  pro¬ 
vide.  The  roles  and  responsibilities  for  IA  certification  and  approval  to  operate  for  the  system 
under  development  are  summarized  below.  The  key  IA  certification  authority  is  the  desig¬ 
nated  approval  authority  (DAA). 

The  DAA  has  the  authority  and  ability  to  evaluate  the  mission,  business  cases,  and  bud¬ 
getary  needs  for  the  system  in  view  of  the  security  risks.  The  DAA  determines  the  acceptable 
level  of  residual  risk  and  makes  the  authorization  decision,  and  it  is  ultimately  responsible  for 
authorizing  or  denying  the  test  or  operation  of  DoD  ISs  (DoD  CIO,  2007,  paragraph  5.14.5). 

Also,  DoD  CIO  (2007,  paragraph  5.14.2)  recognizes  that  the  DAA  is  responsible  for 
ensuring  that  each  DoD  IS  complies  with  applicable  DoD  baseline  IA  Controls  to  intercon¬ 
nect  with  the  GIG. 

The  IA  manager  (IAM)  (also  known  as  the  IS  security  manager  [ISSM])  is  respon¬ 
sible  for  the  IA  program  of  a  DoD  IS  or  organization. 

The  certification  authority  (CA)  manages  the  certification  process.  The  IAM  or  CA 
performs  a  comprehensive  evaluation  of  the  technical  and  nontechnical  aspects  of  the  certifi¬ 
cation  effort,  reports  the  status  of  the  certification,  and  recommends  to  the  DAA  whether  to 
authorize  the  system. 

The  PM  or  system  manager  (SM)  represents  the  interest  of  the  system  throughout  the 
life  cycle  and  assigns  an  IAM. 

The  user  representative  is  concerned  with  system  availability,  integrity,  and  confidential¬ 
ity  as  they  relate  to  the  system’s  mission. 
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Implications  of  Internet  Protocol  version  6  (IPv6)  for 

DoD  I  A  Policy 


•  In  2003,  DoD  stated  that,  as  of  October  1,  2003,  DoD  products  and  systems  will  be 
capable  of  operation  in  Internet  Protocol  version  4  (IPv4)  and  IPv6  networks  and  that 
all  DoD  networks  will  use  IPv6  by  FY  2008.* 

-  Provide  3.4x1 038  IP  addresses  (or  6.5x1 023  addresses/m2  of  the  earth’s  surface) 

•  Eliminate  need  for  Network  Address  Translation 

-  Integrated  IP  security  will  improve  security  for  some  functions 

-  Improved  autoconfiguration  for  mobile  networks 

•  Findings: 

-  IA  standards  for  IPv6  are  still  evolving 

-  Overlay  of  IPv4  and  IPv6  opens  new  potential  security  issues 

-  Military  services  will  not  be  able  to  meet  2008  timeline 

-  IPv6-enabled  mobile,  ad  hoc  networks  will  stress  concepts  of  enclaves  and  IA  management 

•  Each  soldier  is  an  enclave  with  possibly  multiple,  different  MAC/CL  networks 


Current  MANET 


RAND 


*DITO  (2003) 


IPv6-enabled  MANET 
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We  examined  the  implications  of  IPv6  for  DoD  IA  policy.  We  found  that  IA  standards  for 
IPv6  are  still  evolving  and  therefore  cannot  be  fully  assessed  at  this  time.  It  appears,  however, 
that  the  simultaneous  use  of  IPv4  and  IPv6  in  the  same  network  could  potentially  invoke 
additional  security  concerns.  At  this  time,  there  is  also  considerable  uncertainty  as  to  whether 
the  military  services  will  be  able  to  meet  the  DoD  mandate  that  all  DoD  networks  use  IPv6 
by  FY  2008.  One  primary  issue  is  that  IPv6-enabled,  mobile,  ad  hoc  networks  could  stress  the 
concepts  of  enclaves  and  IA  management  because  each  soldier  could  be  considered  an  enclave 
and  each  soldier  could  conceivably  be  associated  with  multiple,  distinct  networks. 
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Integrated  Architectures 

•  Overarching  guidance  on  selecting  and  using  architectures  is  insufficient: 

-  No  definitive  list  of  authoritative  architectures 

-  No  clear  authorities  for  determining  which  architectures  should  be  used 

-  Architectural  framework  may  not  be  well  suited  to  GIG  SOA  approach 

-  Recommendation:  Identify  joint  and  service  architectures  that  should  be  point  of 
reference  for  addressing  particular  mission  areas;  identify  appropriate  joint  and 
service  authorities  that  can  direct  PMs  to  use  particular  architectures 

•  Maintenance  and  application  of  architectures  are  very  difficult 

-  Architectural-view  definitions  permit  significant  variation  in  what  and  how 
information  is  captured 

-  Significant  ambiguities  are  common,  especially  for  data  interoperability 

-  Architectural-view  formats  and  packages  are  not  compatible  with  each  other 

-  Static  views  are  difficult  to  employ  directly,  much  less  integrate,  and  metrics  for 
assessing  integrated  architectures  are  limited 

-  Recommendation:  Revise  architectural-view  definitions  to  provide  precise 
expectations  of  how  products  should  look  based  on  “best”  architectures 
developed 

•  Views  should  be  sufficient  to  define  data  interoperability  and  IA  controls 

-  Recommendation:  Consider  developing  automated  architecture  standards  and 
eventually  requiring  compliance  with  these  standards  (e.g.,  UML  2.0  /  SysML) 

•  Specifying  a  common  architectural  model  exchange  format  would  be 

RAND  particularly  useful  38 


This  slide  summarizes  several  issues  associated  with  the  definition  and  development  of  inte¬ 
grated  architectures. 
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